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Introduction 


_ .  Jackknifing  techniques  are  increasingly  being  applied  to  data 

analysis  for  bias  reduction.  In  robust  estimation,  several  studies 
have  recently  been  published  giving  asymptotic  properties  of  jack¬ 
knifed  estimates,  Cheng  (1982)  has  demonstrated  the  validity  of 
jackknifing  L-estimates  under  various  conditions  on  the  score  function, 
Efron  (1982)  has  shown  that  jackknife  turns  out  to  be  a  special  case 
of  his  bootstrap  technique. 

In  problems  of  density  estimation,  improvement  of  kernel  type 
estimates  was  proposed  by  Schucany  and  Sommer  (1977)  through  the 
technique  of  combining  several  estimates  using  different  kernels. 
Usually  it  is  possible  to  reduce  bias  in  kernel-type  estimates  simply 
by  a  judicious  choice  of  a  kernel.  However,  in  that  case,  the 
estimates  of  density  functions  can  be  negative.  The  situation  has  been 
described  by  Stute  (1982)  in  his  paper  showing  that  the  use  of  non¬ 
negative  kernels  does  not  allow  the  possibility  of  reduction  of  bias. 

In  this  paper,  the  effect  of  jackknifing  using  leave-out  rules, 
is  studied.  i’seudovalues  in  case  of  density  estimates  are  defined  and 
optimal  properties  of  the  jackknifed  estimates  are  given.  It  is  shown 
that  the  asymptotic  behavior  of  the  jackknifed  estimates  is  the  same 
as  that  of  the  classical  estimate,  A  Berry-Esseen  type  central 
limit  theorem  showing  the  normality  of  the  jackknifed  estimate  is 


also  given. 
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2 .  Pseudovalues 

Let  X^,  X2 , . . . ,Xn  be  a  random  sample  from  a  population  with 
cumulative  distribution  function  F(x)  and  probability  density 
function  f(x).  Let  K(x)  be  a  given  kernel  function  with  the 
following  properties. 


P  (i) 

sup  |  K(x)  |  <  00 

P  (ii) 

/  K(x)dx  =  1 

? (iii) 

lim  jxK(x) |  = 

X-KX> 

0 

P(iv) 

/  X1  K(x)dx  - 

_oo 

0,  i  -  1, 

and  J|xr  K(x) 

|  dx  <  00 

Let  Fn(x)  be  the  empirical  distribution  function 
random  sample  and  let  h^  be  a  sequence  of  constants, 
kernel  density  estimates  of  f (x) ,  defined  by  Rosenblat 
Parzen  (1962)  are  given  by 


A  n  x-X. 

'„<»)  -  <»  M"1  I 


i=l 


based  on  the 
Then  the 
t  (1956)  and 


=  C  /  dF(y) 


(2.1) 


It  is  well  known  that  the  expectation  of  fn(x) > 
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K[C„(x)l  =  I >“’/  K(~*)dF(y)  -  C(x) 

n 

as  n  ->  U1  and  h  •+  0, 
n 

Also  the  variance  of  f^(x), 

A 

V[f  (x)  ]  4  0  if  in  addition  n  h  The  above  results  can  be 

n  n 

found  in  an  extensive  survey  of  probability  density  estimation  by 
Tapia  and  Thompson  (1978) . 

Let  ^(x)  is  the  empirical  distribution  function  of  the  random 
sample  X^,.,.,X  with  the  observation  removed  and  let 

(x)  *  ~ —  /  K(^-x~^)dF^  ^(y)  where  h^  ^  is  a  sequence  of  constants 
n-1  n-1 

based  on  n-1  observations. 

Define  the  pseudovalues  as  follows. 


M 

fs(x) 


fn(x) 


(2.2) 


The  jackknifed  estimate  of  the  probability  density 
defined  by  the  following 

a  i  n  a  . 

fJ(x)  =  n  fs(x) 
i=l 


function  is  then 


(2.3) 


In  this  section,  several  properties  of  the  Jackknifed  estimates 
are  discussed  especially  its  bias  redaction  property.  The  difficulty 
of  bias  reduction  without  negative  kernels,  has  been  demonstrated  by 
several  authors,  e.g.  Stute  (1982,  p.  419) .  Notice  that  for 
sufficiently  smooth  F's,  it  is  always  possible  to  reduce  the  bias 

E[ f  (t) ]-f(t),  by  choosing  appropriate  kernels  K.  Also  amone  the 
class  of  nonnegative  K's,  P(iv)  can  be  achieved  only  for  r  =  1,  thus 

giving  h^  as  the  best  possible  error  rate.  For  better  results,  one 
has  to  include  also  these  K’s  for  which  K(y)  may  be  negative,  leading 
to  an  estimate  of  fn(t)  which  may  be  negative. 

From  now  on,  we  shall  assume  that  kernels  satisfy  the  following 
additional  properties, 

P(v)  The  rth  derivatives  of  density  function  satisfy  a 

Lipshitz  condition 

|  f  (x)-f  ^  (y)  |  ■'  c  |x-y  I’1,  0  ^  a  1 

for  all  x,  and  y. 

P(vi)  {h^}  ■*-s  a  se9uence  constants  satisfying 

h 

r-5-  =  1  +  o(l), 


P(vii)  /|xr+a  K(x)|dx  < 


We  prove  the  following  theorem. 


Theorem  3.1.  Under  the  conditions  P(i)  -  P(vii)  on  kernel  K(x) , 


hrf^(x)  /  zrK(-z)dz 
n  J 


E[fn(x)J  =  f(x)  + 


r ! 


+  °  o  • 


Proof . 


E[f  (x)]  =  /  K(-z)f(x+zh  )dz 


_0O 


f(x)  +  /  K(-z) [ f (x+zh^)-f (x) ]dz . 


(3. 


Using  the  Taylor's  expansion  with  integral  remainder,  we  have, 

r.  r-1 

7.  h  . 

f(x+zhn)  =  f  (x)  +  zh^f'Cx)  +  ...  +  "^ITyr  r  r  ^  (x) 


+  / 
X 


x+zh  (x+zh  -O 
n  n 


r-1 


(r-1) 


f(r)(OdC. 


(3. 


Using  P(iv),  we  have 


~  ™  x+zh  (x+zh  -f.) 

E  l  f n  (x)  ]  =  f  (x)  +  /  /  n  - 

_oo  x 


C  -  ] 


(r-1)! 


f (r)  ('JK(-z)d'.dz  (3. 


Let 


00  x+zh  (x+zh  -£) 

If  n  ~ 

-CO  x 


r-1 


(r-1)! 


[f(r)  (f.)-f(r)  (x)+f(r)(x)]K(-z)d£dz 


f(r)(x) 

r! 


/  K(-z)(zh)rd  + 


x+zh  (x+zh  -£) 

f  f  U  -  -  rA-r-, 


r-1 


[f(r)(£)-f(r)(x),JK(-z)dCdz 


(3. 


The  second  integral  on  the  right  of  (3.4)  in  absolute  value  is 


"  x+zh  (x+zh  -O 

<  c  /  /  n  n 

00  x 


r-1 


( r- 1)"! -  K(-z) dCdz  using  P(v) 


00  n  x+zh  (x+zh  -£) 

<  c  /  |K(-z)||zhn|a  /  n  - 7“]yr 

00  x  1  ■ 


r-1 


d£  dz 


—  /  I K(-z) | | zh^ | r+adz  by  P(vii) 


-  o(hr+a). 


(3.5: 


Combining  (3.4)  and  (3.5),  we  get  the  result  (3.1)  proving  the 
theorem.  !~1 


Using  the  results  of  Theorem  3.1,  we  can  find  the  bias  of  the 


jackknifed  estimate  fj(x).  The  result  is  given  in  Theorem  3.2. 


Theorem  3.2.  Under  the  conditions  P(i)  -  P(vii)  for  the  kernel. 


the  bias  of  the  jackknifed  estimate  fj(x)  is 


r+1 


0(- 


(h  -h  ) 
n-i  n 


1-a 


(3.6 


Proof.  The  expectation  of  the  jackknifed  estimate,  from  equation 


(2.3),  is  given  by 
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ElfjW]  -  Ov-'Vl'  tVEIfn<’‘)l 

-r  "i 


"  hii-lE[fn-l(x)  1 1 


Bias  fj(x)  =  E(f j(x) )  -  f(x) 


=  (h  r  -h*'r1 )  Mh^  Bias  f  (x) 
n  n— x  n  n 

A  , 

-h  r.  Bias  f1  (x)}  (3-7) 

n-i.  n-i 


Using  (3.3),  we  have 


r-1 


Bias  fj(x)  =  r  fZhn  fCr,(OK(-2)d5  dz 

x 


°°  x+zh  (x+zh  .-£) 
~r  t  r  n-1  _  n-1 

-  hn-l  J  / 

_CD  X 


r-1 


(r-1)! 


f(r)(QK(-z)d£dz 


(3.8) 


Making  the  transformation 


?  -  x  =  z  h  n 
'  n 

in  the  first  integral  and  f,  -  x  =  zh  In  the  second  integral  in 


(3.8),  we  have 
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-  (h“r-h  r1)”1 

,  „  n  n-1 
Bias  fj(x)  =  ~ (TTYyj - 


/  /  (1-n)  [f^(x+n  h  z)-f U;  (x+nhn_1z)  ]K(-z)zrdr|  dz 

oo  o 

(h~r-h~r  )'1  <*> 

-  "T^lf T —  ^  /  (i-n)r"1na(hn-hn_1)clza+rK(-2)dn  dz 


r(r) 


=  0( 


^hn~hn-l) 

h"r-h  r, 
n  n-1 


a 


h  ^-h 

Assuming  - ^ -  =  o(l) ,  we  have  the  bias  reduced  to 

n 


r+1 


0(- 


(h  -h  ) 
n-1  n 


1-a 


since 


h  h  .  -h 

G-n-;'  .  a  - 


n-1 


h  .  -h 

4  1  -  . -) 

n 


Remark.  The  comparison  of  theorem  3.2  with  theorem  3.1  clearly 


demonstrates  the  reduction  of  bias  of  the  jackknifed  estimate  f^(x) 

by  at  least  of  the  term  h  .  By  the  proper  choice  of  h  .  we  can 

n  n  ’ 

reduce  the  second  terra  also  under  certain  smoothness  conditions 
on  the  probability  density  function  f. 


Variance  of  f^(x) 

Using  the  expressions  for  jackkifed  estimate  in  (2.2)  and  (2.3) 
we  have  the  variance  of  the  estimate,  Oj  as  follows; 
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o]  *  n“1(h~r-h’'r.)~2  Varfh^K^) 
J  n  n— i  n  n 


’'T-l, 


n-1 


A  +  B 


where 


-  hn-llx(ir1^)l!f<y,d)' 

n-1 


n 

-  C^rV'w^ 

n-1 


Notice  that  with  z  -  (x-y)hn‘,  we  have 


A  =  n’1(h~r-h"r1)"2h*'2r“1 
n  n-1  n 

~  h  .  h 

/  (K(z)-(r-S~)r+iK(z  .-&-)] 2f(x-zh  )dz 
_  „  n  (i  ,  n 

03  n-1  n-1 


(n  h  )"‘(h  r-h  r1)'/h'2Y(l-  r—1-)1' 
n  n  n-1  n  h  . 

h  hn-L 

C  K(z)-K(z  t^)  K(z 

/  {t - azL.j  .  . — [(-Ji-)r+1- 

i  h  J  h  h  / 

n  ,  n  n-1 


1  - 


1  - 


n-1 


n-1 


f (x  -  z  h^Jdz . 


1]}? 
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In  the  limit  when  t—”-  1  •  n  h  -*■  *»*,  h  -*•  0  as  n  ■*  ", 

h  '  n  1  n  r 


n-1 


n  n 


lim(n  h^A)  -  f(x)  /  (z  K1 (z)+K(z) (r+1)) 2dz  , 


n-«° 


so  that 


CO 

A  "  ;nr^  ^  lz  Rl(z)  +  K(z><r+1>J2,lz  +  0(-^r)  • 


n  h 


We  use  the  conditions  that  /z2[  K’ (z)j2dz  <  ®  and  /  K2(z)  «  0(m~2) 

m 

Also  note  that  from  theorem  (3,2),  we  have 


J[f<x)  +  0( 


r+1 

i 

n 


(h  .-h  ) 
n-1  n' 


1-u 


) 


Hence,  we  have,  as  n  + 

CO 

a2  *  /  U  K’(z)  +  (r+l)K(z)  }2dz 

>  n  h  1 


n 


+  o 

n  h 

n 


as  B  contains  terms  of  much  lower  order 
Notice  that  >  0  since  zK'(z)  + 
functions  K*(z).  If  zK'(z)  +  (r+l)K(z) 


than  <>(n  1  !i  *). 

n 

(r+l)K(z)  [  0  for  a 
=  0,  then  K(z) 


11  integrable 
-  z-(r+1) 


which  is  not  integrable. 


4 .  Central  Limit  Theorem  for  the  Jackknifed  Estimate 

Since  the  estimate  is  a  sum  of  n  independently  distributed  random 
variables,  the  following  theorem  can  be  proved  under  the  usual 
conditions  of  central  limit  theorems  since  f^(x)  and  similarly 
f  ^(x)  are  asymptotically  normal. 

Theorem.  As  n  ->  <«, 

f,(x)-f(x) 

Pr  { - - -  <  y>  -*•  4>(y) 

J 


To  find  the  Berry-Esseen  bounds,  we  need  (2+<$)-th  moment  of 
the  jackknifed  estimate  (2,2)  which  is  an  average  of  f<,(::),  i=l,2,.,t, 
given  by  (2.1),  Using  the  expresssion s  in  terms  of  kernel  K  for 

A  /  ,  v  A  j  A 

f;  (x)  and  f  . (x) ,  we  can  write  the  (2+6)-th  moment  of  f.(x).  By 
n  n  l  j 

Jensen's  inequality,  we  have 


,  2+6  2+6 
y2+6  =  E|fJ(x)-EfJ(x)|  <  A2+6{E|f^(x)| 


2+6 

+  |E(fg(x))|  } 


i 2  V6EI£S(*>I  > 


where  A^+(j  is  constant  depending  on  6  and 

^ ,  2+6  2+5 

(E|fJ(x)|)  <E|f£(x)|  . 

.  2+6 
Hence  p~  .  ,  —  Eu^L[.  <  2nA~  .  r  E I  f  „  (x)  I 


E|fg<X)  | 


n  n-1 


< 


C 
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Fcr  estimation  of  p7+^,  we  use  the 

«.VI 

t  2+6 .  ,  .  .  .  2+  A . 

j  z  (K' (z) )  dz  v 


conditions. 


/  K2+5(z)dr  = 

m  m 


Now  we  state  the  Berry-Esseen  type  theorem  for  fj(x),  For  reference 
see  Chao  and  Teicher  (1978,  p.  299). 


Theorem  4,1 


n(f j(x)-Ef T(x)) 

sup|p{ - - -  <  x)  -  4>(x)|  £ 


<  C 


^2+6 

6  2+6 

o. 


The  above  result  gives  the  uniform  convergence  of  the  central  limit 
theorem  for  the  jackknifed  density  estimate. 
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